Synthesizing Free-Viewpoint Images from Multiple View Videos in Soccer Stadium

We propose a new method for synthesizing freeviewpoint images from multiple view videos in soccer stadium. The previous method[1] can synthesize virtual viewpoint images at only intermediate viewpoint between real cameras. The previous method estimates only pixel-wise correspondences between input images for synthesizing virtual viewpoint images without a 3D reconstruction. In this paper, we propose a new method that can synthesize virtual viewpoint images at free-viewpoints which are not limited at intermediate viewpoint by reconstructing simple 3D models. To reconstruct the object soccer scene, we perform camera calibration without marker objects because we can save labor to put such objects on the soccer ground. For reconstructing of dynamic regions, we employ a billboard representation of dynamic regions. The position of the billboard is determined by tracking the players. The billboard representation can realize a simple 3D reconstruction and transform textures into the appropriate appearance of the aspect of dynamic regions at virtual viewpoint. We also introduce the choice of optimal cameras for rendering on the billboard. We choose the optimal camera that has the most similar texture at virtual viewpoint. By applying our method to a soccer scene, we can synthesize free-viewpoint images by employing only pixel values of real images. Keywords— image processing, free-viewpoint, multiple view videos, homography, projective matrix, calibration